VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change

Abstract Visual place recognition (VPR) is the process of recognising a previously visited using visual information, often under varying appearance conditions and viewpoint changes with computational constraints. VPR related to concepts localisation, loop closure, image retrieval critical component many autonomous navigation systems ranging from vehicles drones computer vision systems. While concept has been around for years, research grown rapidly as field over past decade due improving camera hardware its potential deep learning-based techniques, become widely studied topic in both robotics communities. This growth however led fragmentation lack standardisation field, especially concerning performance evaluation. Moreover, notion illumination invariance techniques largely assessed qualitatively hence ambiguously past. In this paper, we address these gaps through new comprehensive open-source framework assessing dubbed “VPR-Bench”. VPR-Bench (Open-sourced at: https://github.com/MubarizZaffar/VPR-Bench ) introduces two much-needed capabilities researchers: firstly, it contains benchmark 12 fully-integrated datasets 10 secondly, integrates variation-quantified dataset quantifying invariance. We apply analyse popular evaluation metrics communities, discuss how different complement and/or replace each other, depending upon underlying applications system requirements. Our analysis reveals that no universal SOTA technique exists, since: (a) state-of-the-art (SOTA) achieved by 8 out on at least one dataset, (b) community does not necessarily yield other given differences metrics. Furthermore, identify key open challenges (c) all suffer greatly perceptually-aliased less-structured environments, (d) variance where lateral change less effect than 3D change, (e) directional more adverse effects matching confidence uniform change. also present detailed meta-analyses regarding roles ground-truths, platforms, application requirements parameters. Finally, provides unified implementation deploy datasets, extensible templates.